Skip to content

Comments

Update hca-bone-marrow.Rmd#7

Merged
PeteHaitch merged 2 commits intodevelfrom
hca-bone-marrow-fastmnn
Oct 10, 2025
Merged

Update hca-bone-marrow.Rmd#7
PeteHaitch merged 2 commits intodevelfrom
hca-bone-marrow-fastmnn

Conversation

@PeteHaitch
Copy link
Collaborator

  • Note that MNN-corected values may vary across systems due to the randomized SVD
  • Make cluster.choice robust to these differences
  • This was verified on Ubuntu (x86_64) and macOS (aarch64)

@LTLA is this variation in fastMNN results across systems unavoidable?

- Note that MNN-corected values may vary across systems due to the randomized SVD
- Make `cluster.choice` robust to these differences
- This was verified on Ubuntu (x86_64) and macOS (aarch64)
@LTLA
Copy link
Collaborator

LTLA commented Oct 8, 2025

The offending code is probably something in multiBatchPca() that does an SVD somewhere using svd(), which in turn depends on the BLAS/LAPACK that R was configured to use. There isn't an easy way to avoid this if we want to continue using randomized SVD.

I'd suggest just hacking together a solution to get the book to compile for now. I just finished the first draft of the scrapper book, which will eventually allow us to deprecate some of the old OSCA content, as discussed on Zulip. A lot of these workflow chapters will no longer be necessary as the functions are so fast that we can just repeat all the dataset-specific set-up steps in each chapter.

On that note, scrapper uses Eigen to handle the core SVD so should be independent of the BLAS/LAPACK configuration. However, exact floating-point reproducibility across machines is difficult to guarantee as implementations always have some wiggle room for precision, e.g., FMA instructions, differences in library functions like log(). Eigen itself changes some of its accumulation functions according to whether SIMD is available, so you could get slightly different sums across different machines... that's just how it is.

@PeteHaitch
Copy link
Collaborator Author

Thanks, Aaron.

I'd suggest just hacking together a solution to get the book to compile for now.

That's what I've gone for with this PR, so I'll merge and push.

@PeteHaitch PeteHaitch merged commit 318c3ab into devel Oct 10, 2025
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants